Agenda

Reminders

  • Midterm is June 9\(^{th}\) in class
    • Review is posted on Canvas
    • Let me know if you have any questions or want solutions to previous assignments
  • Final Project Ideas + Groups due ———-
    • need a “solid” idea of what you might look at
    • need data sets, or ideas for data sets
      • these do not need to be your final data sets, but I just want to see you have been thinking about this project
    • You can work by yourself or with a partner
    • Fill out the canvas “quiz” to let me know if you want to work by yourself or with someone else.
    • If you want to work with a partner, but don’t know who to work with, let me know and I will find someone

PackageName::FunctionName

library(tidyverse)
head(diamonds)


#?filter

#filter from dplyr
diamonds %>%
  dplyr::filter(color == "E") %>%
  head()

#normal
diamonds %>%
  filter(color == "E") %>%
  head()
NA

Building Graphics

  1. Draw by hand (or imagine) the specific plot that you intend to construct
  2. Data Wrangling (if needed) to get the data in glyph-ready form, or verify that the current form is glyph-ready for your purposes.
  3. Establish the frame using a ggplot() statement
  4. Create the intended glyph using geom_[style]() such as
    • geom_point()
    • geom_bar()
    • geom_boxplot()
    • geom_density()
    • geom_vline()
    • geom_segment()
    • geom_histogram()
    • and many more
  5. Map variables to the graphical attributes of the glyph using: aes( )
  1. Add additional layers to the frame using the + symbol
    • Note: not %>% between layers of ggplot2 graphics
    • Think + is equivalent of “add layer on top of …” in ggplot2 portions, whereas %>% is “and then the next step is…”

Steps 4 and 5 can be switched.

Example: Baby Names

Let’s look at our BabyNames names data set agian.

# data intake
data("BabyNames", package = "dcData")

# inspect data intake
glimpse(BabyNames)
Rows: 1,792,091
Columns: 4
$ name  <chr> "Mary", "Anna", "Emma", "Elizabeth", "Minnie", "Margaret", "Ida", "…
$ sex   <chr> "F", "F", "F", "F", "F", "F", "F", "F", "F", "F", "F", "F", "F", "F…
$ count <int> 7065, 2604, 2003, 1939, 1746, 1578, 1472, 1414, 1320, 1288, 1258, 1…
$ year  <int> 1880, 1880, 1880, 1880, 1880, 1880, 1880, 1880, 1880, 1880, 1880, 1…

wrangle into glyph-ready form

names <- c("Olivia", "Zoe", "Quentin")

Names <- 
  BabyNames %>%
  filter(name %in% names) %>%
  group_by(name, year) %>%
  summarise(total = sum(count, na.rm = TRUE))
`summarise()` has grouped output by 'name'. You can override using the `.groups` argument.
Names %>%
  head()
NA

in the beginning you might use mplot to get started–here’s the default result

The graph looks perfectly fine, but this code isn’t easy to read.

This is why we stress writing readable code!

ggplot(data = Names, aes(x = year, y = total)) + geom_line()  + aes(colour = name) + theme(legend.position = "right") + labs(title = "")

we can do better

  1. establish the frame
  2. plot the glyphs (i.e., select a geom)
  3. map the aesthetics
  4. add labels and title
  5. other features (e.g., alpha, sizing, etc)

Our Plot

  1. Establish the Frame

Nothing is here! That is exactly what is supposed to happen. Calling ggplot() only tells us R that we are ready to plot and I want to create some space to make my plot.

ggplot(data = Names) 

NA
  1. plot the glyphs (i.e., select a geom)

Still Nothing! We need to tell it what our axis are.

Note that ggplot uses +, NOT %>%. This is because we are adding layers to our plots.

ggplot(data = Names) + 
  geom_line()
Error in `geom_line()`:
! Problem while setting up geom.
ℹ Error occurred in the 1st layer.
Caused by error in `compute_geom_1()`:
! `geom_line()` requires the following missing aesthetics: x and y
Backtrace:
  1. base (local) `<fn>`(x)
  2. ggplot2:::print.ggplot(x)
  4. ggplot2:::ggplot_build.ggplot(x)
  5. ggplot2:::by_layer(...)
 12. ggplot2 (local) f(l = layers[[i]], d = data[[i]])
 13. l$compute_geom_1(d)
 14. ggplot2 (local) compute_geom_1(..., self = self)

Note - this is why I like to map aesthetics first, so we can avoid errors.

  1. Map the aesthetics

Rule of thumb: anytime when you are plotting with ggplot, ALL variables need to be inside an aes (except facets, later in slides), and all constants go outside of the aes.

#not Quite
ggplot(data = Names) + 
  geom_line( aes(x = year, y = total)) 


#add groups
ggplot(data = Names) + 
  geom_line( aes(x = year, y = total, group = name)) 


#add color
ggplot(data = Names) + 
  geom_line( aes(x = year, y = total, color = name)) 

NA
NA
  1. Add labels and title
ggplot(data = Names) + 
  geom_line( aes(x = year, y = total, color = name)) +
  ggtitle("Names Over Time") +
  xlab("Year") +
  ylab("Popularity") +
  guides(color = guide_legend(title = "Siblings Names" ))

NA
NA
  1. other features (e.g., alpha, sizing, etc)
ggplot(data = Names) + 
  geom_line( aes(x = year, y = total, color = name, linetype = name)) +
  ggtitle("Names Over Time") +
  xlim(c(1972, 2022))+
  xlab("Year") +
  ylab("Popularity") +
  guides(color = guide_legend(title = "Siblings Names" ), 
         linetype = guide_legend(title = "Still Siblings Names" ))

NA
NA

Remarks about faceting: facet_wrap()

The syntax for facets requires a formula syntax we haven’t seen much yet. There are two main ways to plot with facets. Here are a few pointers:

Facet Wrap

  • facet_wrap() just makes a seperate plot for each level of the categorical variable
    • Syntax: facet_wrap( ~ categoricalVariable)
    • For example:

Facet Grid

Difference between color and fill

library(mosaicData)

head(CPS85)

CPS85 %>% 
  ggplot() +
  geom_density(aes(x = wage, color = sex), alpha = 0.4)+
  facet_grid( ~ married) +
  xlim(0,30) 

  
CPS85 %>% 
  ggplot() +
  geom_density(aes(x = wage, fill = sex), alpha = 0.4)+
  facet_grid( ~ married) +
  xlim(0,30) 


  
CPS85 %>% 
  ggplot() +
  geom_density(aes(x = wage, fill = sex, color = sex), alpha = 0.4)+
  facet_grid( ~ married) +
  xlim(0,30)




CPS85%>%
  ggplot(aes(x = married, color = sex)) + 
  geom_bar() +
  facet_wrap( ~ union, scales = "free")  #Note the scales here 


CPS85%>%
  ggplot(aes(x = married, fill = sex)) + 
  geom_bar()+
  facet_wrap( ~ union, scales = "free")  #Note the scales here 



CPS85%>%
  ggplot(aes(x = age, y = wage, color = sex)) + 
  geom_point()


CPS85%>%
  ggplot(aes(x = age, y = wage, fill = sex)) +  #fill does not work for points!
  geom_point()

NA
NA

Another Example using Diamonds Data

  1. establish the frame

  2. plot the glyphs (i.e., select a geom)

  3. map the aesthetics

  4. add labels and title

  5. other features (e.g., alpha, sizing, etc)

  6. Establish the Frame

ggplot(data = diamonds)

  1. plot the glyphs (i.e., select a geom)
ggplot(data = diamonds) +
  geom_point()
Error in `geom_point()`:
! Problem while setting up geom.
ℹ Error occurred in the 1st layer.
Caused by error in `compute_geom_1()`:
! `geom_point()` requires the following missing aesthetics: x and y
Backtrace:
  1. base (local) `<fn>`(x)
  2. ggplot2:::print.ggplot(x)
  4. ggplot2:::ggplot_build.ggplot(x)
  5. ggplot2:::by_layer(...)
 12. ggplot2 (local) f(l = layers[[i]], d = data[[i]])
 13. l$compute_geom_1(d)
 14. ggplot2 (local) compute_geom_1(..., self = self)

  1. Map the aesthetics
ggplot(data = diamonds, aes(x = carat, y = price)) +
  geom_point()

  1. Add Titles and Labels
ggplot(data = diamonds, aes(x = carat, y = price)) +
  geom_point(aes(color = depth), alpha = 0.5, size = 1) +
  ggtitle("Diamonds Data") +
  xlab("Carat") +
  ylab("Price")

  1. Add additional features

Notice that I can have aes inside multiple statements. Notice that when I use constants (like alpha = 0.3, size = 0.1) they ARE NOT inside aes.

In general, variables go inside aes and constants go outside of it. (unless we are using facets then see previous materials.)

ggplot(data = diamonds, aes(x = carat, y = price)) +
  geom_point(aes(colour = depth), alpha = 0.3, size = 0.1) +
  ggtitle("Diamonds Data") +
  xlab("Carat") +
  ylab("Price") +
  facet_grid( cut ~ color)


ggplot(data = diamonds, aes(x = carat, y = price)) +
  geom_point(colour = "red", alpha = 0.3, size = 0.1) +
  ggtitle("Diamonds Data") +
  xlab("Carat") +
  ylab("Price") +
  facet_grid( cut ~ color)

Side Note about placement of aes

aes can either go inside the ggplot() function, or inside the geom_[chart]() function itself, or both. The 3 following options create the same plots, but the code is slightly different.

#option 1
ggplot(data = diamonds, ) +
  geom_point(aes(x = carat, y = price, color = clarity),
             alpha = 0.2, 
             size = 1) +
  geom_smooth(method = "glm" , 
              formula = y ~ poly(x, 2),                     # y = b_0 + b_1 x + b_2 x^2 + e
              aes(x = carat, y = price), 
              color = "red") +
  ylim(c(0, 20000))

  


#option 2
ggplot(data = diamonds, aes(x = carat, y = price, color = clarity)) +
  geom_point(alpha = 0.2, 
             size = 1) +
  geom_smooth(method = "glm" , 
              formula = y ~ poly(x, 2),                     # y = b_0 + b_1 x + b_2 x^2 + e
              aes(x = carat, y = price), 
              color = "red") +
  ylim(c(0, 20000))

  

#Option 3
ggplot(data = diamonds,  aes(x = carat, y = price) )+ 
  geom_point( aes(color = clarity),
              alpha = 0.2, 
              size = 1) +
  geom_smooth(method = "glm" , 
              formula = y ~ poly(x, 2),                     # y = b_0 + b_1 x + b_2 x^2 + e
              color = "red") +
  ylim(c(0, 20000))

LS0tCnRpdGxlOiAiTDEwIC0gQmFjayB0byBgZ2dwbG90MmAiCnN1YnRpdGxlOiAiRGF0YSBDb21wdXRpbmcgQ2hhcHRlciA4IgphdXRob3I6IHwKICAgICAgICB8IFByZXNlbnRlcjogT2xpdmlhIEJlY2sKICAgICAgICB8IENvbnRlbnQgY3JlZGl0OiBNYXR0aGV3IEJlY2ttYW4Kb3V0cHV0OiAKICAgIHNsaWR5X3ByZXNlbnRhdGlvbjogZGVmYXVsdAogICAgaHRtbF9ub3RlYm9vazogZGVmYXVsdAotLS0KCgpgYGB7ciBpbmNsdWRlPUZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShkY0RhdGEpCmtuaXRyOjpvcHRzX2NodW5rJHNldCh0aWR5PUZBTFNFLCBtZXNzYWdlPUZBTFNFKQpvcHRpb25zKHdpZHRoID0gODApCmBgYAoKCiMjIEFnZW5kYQoKLSBicmllZiBpbnRyb2R1Y3Rpb24gdG8gYDo6YAotIGBnZ3Bsb3QoKWAgZmx5b3ZlcgotIHJlbWFya3MgYWJvdXQgZmFjZXRzCi0gYGNvbG9yYCB2cy4gYGZpbGxgCgoKIyMjIyBSZW1pbmRlcnMgCgotIE1pZHRlcm0gaXMgSnVuZSA5JF57dGh9JCBpbiBjbGFzcyAKICAtIFJldmlldyBpcyBwb3N0ZWQgb24gQ2FudmFzCiAgLSBMZXQgbWUga25vdyBpZiB5b3UgaGF2ZSBhbnkgcXVlc3Rpb25zIG9yIHdhbnQgc29sdXRpb25zIHRvIHByZXZpb3VzIGFzc2lnbm1lbnRzIAotIEZpbmFsIFByb2plY3QgSWRlYXMgKyBHcm91cHMgZHVlIC0tLS0tLS0tLS0KICAtIG5lZWQgYSAic29saWQiIGlkZWEgb2Ygd2hhdCB5b3UgbWlnaHQgbG9vayBhdCAKICAtIG5lZWQgZGF0YSBzZXRzLCBvciBpZGVhcyBmb3IgZGF0YSBzZXRzIAogICAgLSB0aGVzZSBkbyBub3QgbmVlZCB0byBiZSB5b3VyIGZpbmFsIGRhdGEgc2V0cywgYnV0IEkganVzdCB3YW50IHRvIHNlZSB5b3UgaGF2ZSBiZWVuIHRoaW5raW5nIGFib3V0IHRoaXMgcHJvamVjdCAKICAtIFlvdSBjYW4gd29yayBieSB5b3Vyc2VsZiBvciB3aXRoIGEgcGFydG5lciAKICAtIEZpbGwgb3V0IHRoZSBjYW52YXMgInF1aXoiIHRvIGxldCBtZSBrbm93IGlmIHlvdSB3YW50IHRvIHdvcmsgYnkgeW91cnNlbGYgb3Igd2l0aCBzb21lb25lIGVsc2UuIAogIC0gSWYgeW91IHdhbnQgdG8gd29yayB3aXRoIGEgcGFydG5lciwgYnV0IGRvbid0IGtub3cgd2hvIHRvIHdvcmsgd2l0aCwgbGV0IG1lIGtub3cgYW5kIEkgd2lsbCBmaW5kIHNvbWVvbmUKCiMjIFBhY2thZ2VOYW1lOjpGdW5jdGlvbk5hbWUKCiFbXShpbWFnZXMvcGFja2FnZS1jb25mbGljdHMucG5nKQoKLSBXZSBjYW4gdXNlIHRoZSBgOjpgIGZ1bmN0aW9uIHRvIHJlZmVyZW5jZSBmdW5jdGlvbnMgaW5zaWRlIHBhY2thZ2VzLiAKICAtIFRoaXMgaGVscHMgdXMgYmUgZXh0cmEgc3VyZSB0aGF0IHdlIGFyZSB1c2luZyB0aGUgZXhhY3QgZnVuY3Rpb24gd2Ugd2FudCB0byBiZQogIC0gSGVscHMgYXZvaWQgY29uZmxpY3QgKGkuZS4gd2hlbiAyIHBhY2thZ2VzIHRoYXQgaGF2ZSBhIGZ1bmN0aW9uIHdpdGggdGhlIHNhbWUgbmFtZSB0aGF0IGRvIGRpZmZlcmVudCB0aGluZ3MpCiAgICAtIGBzZWxlY3RgIGlzIGEgdmVyeSBjb21tb24gZnVuY3Rpb24gbmFtZS4gVXN1YWxseSB3ZSBuZWVkIHRvIHVzZSBgZHBseXI6OnNlbGVjdGAKCgoKCmBgYHtyLCBtZXNzYWdlPVRSVUV9CmxpYnJhcnkodGlkeXZlcnNlKQpoZWFkKGRpYW1vbmRzKQoKCiM/ZmlsdGVyCgojZmlsdGVyIGZyb20gZHBseXIKZGlhbW9uZHMgJT4lCiAgZHBseXI6OmZpbHRlcihjb2xvciA9PSAiRSIpICU+JQogIGhlYWQoKQoKI25vcm1hbApkaWFtb25kcyAlPiUKICBmaWx0ZXIoY29sb3IgPT0gIkUiKSAlPiUKICBoZWFkKCkKCmBgYAoKCgoKIyMgQnVpbGRpbmcgR3JhcGhpY3MKCjEuIERyYXcgYnkgaGFuZCAob3IgaW1hZ2luZSkgdGhlIHNwZWNpZmljIHBsb3QgdGhhdCB5b3UgaW50ZW5kIHRvIGNvbnN0cnVjdAoyLiBEYXRhIFdyYW5nbGluZyAoaWYgbmVlZGVkKSB0byBnZXQgdGhlIGRhdGEgaW4gZ2x5cGgtcmVhZHkgZm9ybSwgb3IgdmVyaWZ5IHRoYXQgdGhlIGN1cnJlbnQgZm9ybSBpcyBnbHlwaC1yZWFkeSBmb3IgeW91ciBwdXJwb3Nlcy4KMy4gRXN0YWJsaXNoIHRoZSBmcmFtZSB1c2luZyBhIGBnZ3Bsb3QoKWAgc3RhdGVtZW50CjQuIENyZWF0ZSB0aGUgaW50ZW5kZWQgZ2x5cGggdXNpbmcgYGdlb21fW3N0eWxlXSgpYCBzdWNoIGFzCiAgICAtIGBnZW9tX3BvaW50KClgCiAgICAtIGBnZW9tX2JhcigpYAogICAgLSBgZ2VvbV9ib3hwbG90KClgCiAgICAtIGBnZW9tX2RlbnNpdHkoKWAKICAgIC0gYGdlb21fdmxpbmUoKWAKICAgIC0gYGdlb21fc2VnbWVudCgpYAogICAgLSBgZ2VvbV9oaXN0b2dyYW0oKWAKICAgIC0gYW5kICptYW55KiBtb3JlCjUuIE1hcCB2YXJpYWJsZXMgdG8gdGhlIGdyYXBoaWNhbCBhdHRyaWJ1dGVzIG9mIHRoZSBnbHlwaCB1c2luZzogYGFlcyggKWAKICAtIFJ1bGUgb2YgdGh1bWI6IGFueXRpbWUgd2hlbiB5b3UgYXJlIHBsb3R0aW5nIHdpdGggZ2dwbG90LCBBTEwgdmFyaWFibGVzIG5lZWQgdG8gYmUgaW5zaWRlIGFuIGBhZXNgIChleGNlcHQgZmFjZXRzLCBsYXRlciBpbiBzbGlkZXMpLCBhbmQgYWxsIGNvbnN0YW50cyBnbyBvdXRzaWRlIG9mIHRoZSBgYWVzYC4KICAgIC0gZS5nLiBgZ2VvbV9wb2ludChhZXMoY29sb3IgPSBnZW5kZXIpKWAgdnMuIGBnZW9tX3BvaW50KGNvbG9yID0gInJlZCIpYAogIAo2LiBBZGQgYWRkaXRpb25hbCBsYXllcnMgdG8gdGhlIGZyYW1lIHVzaW5nIHRoZSBgK2Agc3ltYm9sIAogICAgLSBOb3RlOiAqKm5vdCoqIGAlPiVgIGJldHdlZW4gbGF5ZXJzIG9mIGBnZ3Bsb3QyYCBncmFwaGljcwogICAgLSBUaGluayBgK2AgaXMgZXF1aXZhbGVudCBvZiAiYWRkIGxheWVyIG9uIHRvcCBvZiAuLi4iIGluIGBnZ3Bsb3QyYCBwb3J0aW9ucywgd2hlcmVhcyBgJT4lYCBpcyAgImFuZCB0aGVuIHRoZSBuZXh0IHN0ZXAgaXMuLi4iCgoqU3RlcHMgNCBhbmQgNSBjYW4gYmUgc3dpdGNoZWQuKgoKIVtodHRwczovL3R3aXR0ZXIuY29tL3RhbnlhX3NoYXBpcm8vc3RhdHVzLzE1NzY5MzUxNTI1NzUzNDA1NDQ/dD12d2FXOGg2Q0M2MmgwcGt3djluNVlnJnM9MTldKGltYWdlcy9nZ3Bsb3QtY2FrZS5qcGcpCgoKIyMgRXhhbXBsZTogQmFieSBOYW1lcyAKCkxldCdzIGxvb2sgYXQgb3VyIGBCYWJ5TmFtZXNgIG5hbWVzIGRhdGEgc2V0IGFnaWFuLiAKCmBgYHtyfQojIGRhdGEgaW50YWtlCmRhdGEoIkJhYnlOYW1lcyIsIHBhY2thZ2UgPSAiZGNEYXRhIikKCiMgaW5zcGVjdCBkYXRhIGludGFrZQpnbGltcHNlKEJhYnlOYW1lcykKCmBgYAoKCgojIyB3cmFuZ2xlIGludG8gZ2x5cGgtcmVhZHkgZm9ybQoKCmBgYHtyfQpuYW1lcyA8LSBjKCJPbGl2aWEiLCAiWm9lIiwgIlF1ZW50aW4iKQoKTmFtZXMgPC0gCiAgQmFieU5hbWVzICU+JQogIGZpbHRlcihuYW1lICVpbiUgbmFtZXMpICU+JQogIGdyb3VwX2J5KG5hbWUsIHllYXIpICU+JQogIHN1bW1hcmlzZSh0b3RhbCA9IHN1bShjb3VudCwgbmEucm0gPSBUUlVFKSkKCk5hbWVzICU+JQogIGhlYWQoKQoKYGBgCgoKIyMgaW4gdGhlIGJlZ2lubmluZyB5b3UgbWlnaHQgdXNlIG1wbG90IHRvIGdldCBzdGFydGVkLS1oZXJlJ3MgdGhlIGRlZmF1bHQgcmVzdWx0CgpUaGUgZ3JhcGggbG9va3MgcGVyZmVjdGx5IGZpbmUsIGJ1dCB0aGlzIGNvZGUgaXNuJ3QgZWFzeSB0byByZWFkLgoKVGhpcyBpcyB3aHkgd2Ugc3RyZXNzIHdyaXRpbmcgcmVhZGFibGUgY29kZSEgCgpgYGB7cn0KZ2dwbG90KGRhdGEgPSBOYW1lcywgYWVzKHggPSB5ZWFyLCB5ID0gdG90YWwpKSArIGdlb21fbGluZSgpICArIGFlcyhjb2xvdXIgPSBuYW1lKSArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJyaWdodCIpICsgbGFicyh0aXRsZSA9ICIiKQpgYGAKCgojIyB3ZSBjYW4gZG8gYmV0dGVyIAoxLiBlc3RhYmxpc2ggdGhlIGZyYW1lCjIuIHBsb3QgdGhlIGdseXBocyAoaS5lLiwgc2VsZWN0IGEgZ2VvbSkKMy4gbWFwIHRoZSBhZXN0aGV0aWNzCjQuIGFkZCBsYWJlbHMgYW5kIHRpdGxlCjUuIG90aGVyIGZlYXR1cmVzIChlLmcuLCBhbHBoYSwgc2l6aW5nLCBldGMpCgoKIyMjIyBPdXIgUGxvdCAKCjEuIEVzdGFibGlzaCB0aGUgRnJhbWUKCk5vdGhpbmcgaXMgaGVyZSEgVGhhdCBpcyBleGFjdGx5IHdoYXQgaXMgc3VwcG9zZWQgdG8gaGFwcGVuLiBDYWxsaW5nIGBnZ3Bsb3QoKWAgb25seSB0ZWxscyB1cyBSIHRoYXQgd2UgYXJlIHJlYWR5IHRvIHBsb3QgYW5kIEkgd2FudCB0byBjcmVhdGUgc29tZSBzcGFjZSB0byBtYWtlIG15IHBsb3QuIAoKYGBge3J9CmdncGxvdChkYXRhID0gTmFtZXMpIAogIApgYGAKCjIuIHBsb3QgdGhlIGdseXBocyAoaS5lLiwgc2VsZWN0IGEgZ2VvbSkKClN0aWxsIE5vdGhpbmchIFdlIG5lZWQgdG8gdGVsbCBpdCB3aGF0IG91ciBheGlzIGFyZS4gCgpOb3RlIHRoYXQgZ2dwbG90IHVzZXMgYCtgLCBOT1QgYCU+JWAuIFRoaXMgaXMgYmVjYXVzZSB3ZSBhcmUgKiphZGRpbmcqKiBsYXllcnMgdG8gb3VyIHBsb3RzLgoKYGBge3IsIGVycm9yPVR9CmdncGxvdChkYXRhID0gTmFtZXMpICsgCiAgZ2VvbV9saW5lKCkKCmBgYAoKCk5vdGUgLSB0aGlzIGlzIHdoeSBJIGxpa2UgdG8gbWFwIGFlc3RoZXRpY3MgZmlyc3QsIHNvIHdlIGNhbiBhdm9pZCBlcnJvcnMuIAoKCjMuIE1hcCB0aGUgYWVzdGhldGljcwoKUnVsZSBvZiB0aHVtYjogYW55dGltZSB3aGVuIHlvdSBhcmUgcGxvdHRpbmcgd2l0aCBnZ3Bsb3QsIEFMTCB2YXJpYWJsZXMgbmVlZCB0byBiZSBpbnNpZGUgYW4gYGFlc2AgKGV4Y2VwdCBmYWNldHMsIGxhdGVyIGluIHNsaWRlcyksIGFuZCBhbGwgY29uc3RhbnRzIGdvIG91dHNpZGUgb2YgdGhlIGBhZXNgLiAKCgpgYGB7cn0KI25vdCBRdWl0ZQpnZ3Bsb3QoZGF0YSA9IE5hbWVzKSArIAogIGdlb21fbGluZSggYWVzKHggPSB5ZWFyLCB5ID0gdG90YWwpKSAKCiNhZGQgZ3JvdXBzCmdncGxvdChkYXRhID0gTmFtZXMpICsgCiAgZ2VvbV9saW5lKCBhZXMoeCA9IHllYXIsIHkgPSB0b3RhbCwgZ3JvdXAgPSBuYW1lKSkgCgojYWRkIGNvbG9yCiMgbm90ZSB0aGF0IGNvbG9yIGluY2x1ZGVzIHRoZSBncm91cHMgYXJndW1lbnQgYnV0IG5vdCB2aWNlIHZlcnNhISAKZ2dwbG90KGRhdGEgPSBOYW1lcykgKyAKICBnZW9tX2xpbmUoIGFlcyh4ID0geWVhciwgeSA9IHRvdGFsLCBjb2xvciA9IG5hbWUpKSAKCgpgYGAKCgo0LiBBZGQgbGFiZWxzIGFuZCB0aXRsZQoKCmBgYHtyfQpnZ3Bsb3QoZGF0YSA9IE5hbWVzKSArIAogIGdlb21fbGluZSggYWVzKHggPSB5ZWFyLCB5ID0gdG90YWwsIGNvbG9yID0gbmFtZSkpICsKICBnZ3RpdGxlKCJOYW1lcyBPdmVyIFRpbWUiKSArCiAgeGxhYigiWWVhciIpICsKICB5bGFiKCJQb3B1bGFyaXR5IikgKwogIGd1aWRlcyhjb2xvciA9IGd1aWRlX2xlZ2VuZCh0aXRsZSA9ICJTaWJsaW5ncyBOYW1lcyIgKSkKCgpgYGAKCgoKNS4gb3RoZXIgZmVhdHVyZXMgKGUuZy4sIGFscGhhLCBzaXppbmcsIGV0YykKCmBgYHtyfQpnZ3Bsb3QoZGF0YSA9IE5hbWVzKSArIAogIGdlb21fbGluZSggYWVzKHggPSB5ZWFyLCB5ID0gdG90YWwsIGNvbG9yID0gbmFtZSwgbGluZXR5cGUgPSBuYW1lKSkgKwogIGdndGl0bGUoIk5hbWVzIE92ZXIgVGltZSIpICsKICB4bGltKGMoMTk3MiwgMjAyMikpKwogIHhsYWIoIlllYXIiKSArCiAgeWxhYigiUG9wdWxhcml0eSIpICsKICBndWlkZXMoY29sb3IgPSBndWlkZV9sZWdlbmQodGl0bGUgPSAiU2libGluZ3MgTmFtZXMiICksIAogICAgICAgICBsaW5ldHlwZSA9IGd1aWRlX2xlZ2VuZCh0aXRsZSA9ICJTdGlsbCBTaWJsaW5ncyBOYW1lcyIgKSkKCgpgYGAKCgoKCgoKIyMgUmVtYXJrcyBhYm91dCBmYWNldGluZzogYGZhY2V0X3dyYXAoKWAKClRoZSBzeW50YXggZm9yIGZhY2V0cyByZXF1aXJlcyBhIGZvcm11bGEgc3ludGF4IHdlIGhhdmVuJ3Qgc2VlbiBtdWNoIHlldC4gIFRoZXJlIGFyZSB0d28gbWFpbiB3YXlzIHRvIHBsb3Qgd2l0aCBmYWNldHMuICBIZXJlIGFyZSBhIGZldyBwb2ludGVyczoKCiMjIyBGYWNldCBXcmFwCgotIGBmYWNldF93cmFwKClgIGp1c3QgbWFrZXMgYSBzZXBlcmF0ZSBwbG90IGZvciBlYWNoIGxldmVsIG9mIHRoZSBjYXRlZ29yaWNhbCB2YXJpYWJsZQogICAgLSBTeW50YXg6IGBmYWNldF93cmFwKCB+IGNhdGVnb3JpY2FsVmFyaWFibGUpYAogICAgLSBGb3IgZXhhbXBsZToKCmBgYHtyfQpkYXRhKCJOQ0hTIikKCiMgYCFpcy5uYShzbW9rZXIpYCBmaW5kcyBjYXNlcyB0aGF0IGFyZSBub24tbWlzc2luZyBmb3IgYHNtb2tlcmAgKGkuZS4gcmVtb3ZlcyBOQSdzKQpIZWlnaHRzIDwtIAogIE5DSFMgJT4lCiAgZmlsdGVyKGFnZSA+IDIwLCAhaXMubmEoc21va2VyKSkgJT4lICAgCiAgZ3JvdXBfYnkoc2V4LCBzbW9rZXIsIGFnZSkgJT4lCiAgc3VtbWFyaXNlKGhlaWdodCA9IG1lYW4oaGVpZ2h0LCBuYS5ybSA9IFRSVUUpKQoKaGVhZChIZWlnaHRzKQoKSGVpZ2h0cyAlPiUKICBnZ3Bsb3QoYWVzKHggPSBhZ2UsIHkgPSBoZWlnaHQpKSArICAgCiAgZ2VvbV9saW5lKGFlcyhsaW5ldHlwZSA9IHNtb2tlcikpICsgICAKICBmYWNldF93cmFwKCB+IHNleCkKYGBgCgojIyBGYWNldCBHcmlkCgotIGBmYWNldF9ncmlkKClgIGFsbG93cyBjb250cm9sIG9mIHJvdyAmIGNvbHVtbiBmYWNldHMKLSBgZmFjZXRfZ3JpZCgpYCBzeW50YXg6CiAgICAtIHJvdyAmIGNvbHVtbiBmYWNldHM6IGBmYWNldF9ncmlkKHJvd3MgfiBjb2xzKWAKICAgIC0gcm93IGZhY2V0cyBvbmx5OiBgZmFjZXRfZ3JpZCggcm93cyB+IC4gKWAgKG5vdGUgdGhlIHJlcXVpcmVkICJgLmAiKSAKICAgIC0gY29sdW1uIGZhY2V0cyBvbmx5OiBgZmFjZXRfZ3JpZCggfiBjb2xzKWAgKG5vICJgLmAiIHRoaXMgdGltZSkKCmBgYHtyfQpIZWlnaHRzICU+JQogIGdncGxvdChhZXMoeCA9IGFnZSwgeSA9IGhlaWdodCkpICsgCiAgZ2VvbV9saW5lKGFlcyhsaW5ldHlwZSA9IHNtb2tlcikpICsgCiAgZmFjZXRfZ3JpZChzZXggfiAuKQpgYGAKCmBgYHtyfQpIZWlnaHRzICU+JQogIGdncGxvdChhZXMoeCA9IGFnZSwgeSA9IGhlaWdodCkpICsgCiAgZ2VvbV9saW5lKCkgKyAKICBmYWNldF9ncmlkKHNleCB+IHNtb2tlcikKYGBgCgoKIyMgRGlmZmVyZW5jZSBiZXR3ZWVuIGBjb2xvcmAgYW5kIGBmaWxsYAoKYGBge3J9CmxpYnJhcnkobW9zYWljRGF0YSkKCmhlYWQoQ1BTODUpCgpDUFM4NSAlPiUgCiAgZ2dwbG90KCkgKwogIGdlb21fZGVuc2l0eShhZXMoeCA9IHdhZ2UsIGNvbG9yID0gc2V4KSwgYWxwaGEgPSAwLjQpKwogIGZhY2V0X2dyaWQoIH4gbWFycmllZCkgKwogIHhsaW0oMCwzMCkgCiAgCkNQUzg1ICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9kZW5zaXR5KGFlcyh4ID0gd2FnZSwgZmlsbCA9IHNleCksIGFscGhhID0gMC40KSsKICBmYWNldF9ncmlkKCB+IG1hcnJpZWQpICsKICB4bGltKDAsMzApIAoKICAKQ1BTODUgJT4lIAogIGdncGxvdCgpICsKICBnZW9tX2RlbnNpdHkoYWVzKHggPSB3YWdlLCBmaWxsID0gc2V4LCBjb2xvciA9IHNleCksIGFscGhhID0gMC40KSsKICBmYWNldF9ncmlkKCB+IG1hcnJpZWQpICsKICB4bGltKDAsMzApCgoKCkNQUzg1JT4lCiAgZ2dwbG90KGFlcyh4ID0gbWFycmllZCwgY29sb3IgPSBzZXgpKSArIAogIGdlb21fYmFyKCkgKwogIGZhY2V0X3dyYXAoIH4gdW5pb24sIHNjYWxlcyA9ICJmcmVlIikgICNOb3RlIHRoZSBzY2FsZXMgaGVyZSAKCkNQUzg1JT4lCiAgZ2dwbG90KGFlcyh4ID0gbWFycmllZCwgZmlsbCA9IHNleCkpICsgCiAgZ2VvbV9iYXIoKSsKICBmYWNldF93cmFwKCB+IHVuaW9uLCBzY2FsZXMgPSAiZnJlZSIpICAjTm90ZSB0aGUgc2NhbGVzIGhlcmUgCgoKQ1BTODUlPiUKICBnZ3Bsb3QoYWVzKHggPSBhZ2UsIHkgPSB3YWdlLCBjb2xvciA9IHNleCkpICsgCiAgZ2VvbV9wb2ludCgpCgpDUFM4NSU+JQogIGdncGxvdChhZXMoeCA9IGFnZSwgeSA9IHdhZ2UsIGZpbGwgPSBzZXgpKSArICAjZmlsbCBkb2VzIG5vdCB3b3JrIGZvciBwb2ludHMhCiAgZ2VvbV9wb2ludCgpCgoKYGBgCgoKCgoKCgoKCiMjIEFub3RoZXIgRXhhbXBsZSB1c2luZyBEaWFtb25kcyBEYXRhCgoxLiBlc3RhYmxpc2ggdGhlIGZyYW1lCjIuIHBsb3QgdGhlIGdseXBocyAoaS5lLiwgc2VsZWN0IGEgZ2VvbSkKMy4gbWFwIHRoZSBhZXN0aGV0aWNzCjQuIGFkZCBsYWJlbHMgYW5kIHRpdGxlCjUuIG90aGVyIGZlYXR1cmVzIChlLmcuLCBhbHBoYSwgc2l6aW5nLCBldGMpCgoxLiBFc3RhYmxpc2ggdGhlIEZyYW1lCgpgYGB7cn0KZ2dwbG90KGRhdGEgPSBkaWFtb25kcykKYGBgCgoyLiBwbG90IHRoZSBnbHlwaHMgKGkuZS4sIHNlbGVjdCBhIGdlb20pCgpgYGB7ciwgZXJyb3IgPSBUUlVFfQpnZ3Bsb3QoZGF0YSA9IGRpYW1vbmRzKSArCiAgZ2VvbV9wb2ludCgpCgpgYGAKCjMuIE1hcCB0aGUgYWVzdGhldGljcwoKYGBge3J9CmdncGxvdChkYXRhID0gZGlhbW9uZHMsIGFlcyh4ID0gY2FyYXQsIHkgPSBwcmljZSkpICsKICBnZW9tX3BvaW50KCkKCmBgYAoKCjQuIEFkZCBUaXRsZXMgYW5kIExhYmVscyAKCmBgYHtyfQpnZ3Bsb3QoZGF0YSA9IGRpYW1vbmRzLCBhZXMoeCA9IGNhcmF0LCB5ID0gcHJpY2UpKSArCiAgZ2VvbV9wb2ludChhZXMoY29sb3IgPSBkZXB0aCksIGFscGhhID0gMC41LCBzaXplID0gMSkgKwogIGdndGl0bGUoIkRpYW1vbmRzIERhdGEiKSArCiAgeGxhYigiQ2FyYXQiKSArCiAgeWxhYigiUHJpY2UiKQoKYGBgCgoKNS4gQWRkIGFkZGl0aW9uYWwgZmVhdHVyZXMgCgoKTm90aWNlIHRoYXQgSSBjYW4gaGF2ZSBgYWVzYCBpbnNpZGUgbXVsdGlwbGUgc3RhdGVtZW50cy4gTm90aWNlIHRoYXQgd2hlbiBJIHVzZSBjb25zdGFudHMgKGxpa2UgYGFscGhhID0gMC4zLCBzaXplID0gMC4xYCkgdGhleSBBUkUgTk9UIGluc2lkZSBgYWVzYC4gCgpJbiBnZW5lcmFsLCB2YXJpYWJsZXMgZ28gaW5zaWRlIGBhZXNgIGFuZCBjb25zdGFudHMgZ28gb3V0c2lkZSBvZiBpdC4gKHVubGVzcyB3ZSBhcmUgdXNpbmcgZmFjZXRzIHRoZW4gc2VlIHByZXZpb3VzIG1hdGVyaWFscy4pCgpgYGB7cn0KZ2dwbG90KGRhdGEgPSBkaWFtb25kcywgYWVzKHggPSBjYXJhdCwgeSA9IHByaWNlKSkgKwogIGdlb21fcG9pbnQoYWVzKGNvbG91ciA9IGRlcHRoKSwgYWxwaGEgPSAwLjMsIHNpemUgPSAwLjEpICsKICBnZ3RpdGxlKCJEaWFtb25kcyBEYXRhIikgKwogIHhsYWIoIkNhcmF0IikgKwogIHlsYWIoIlByaWNlIikgKwogIGZhY2V0X2dyaWQoIGN1dCB+IGNvbG9yKQoKZ2dwbG90KGRhdGEgPSBkaWFtb25kcywgYWVzKHggPSBjYXJhdCwgeSA9IHByaWNlKSkgKwogIGdlb21fcG9pbnQoY29sb3VyID0gInJlZCIsIGFscGhhID0gMC4zLCBzaXplID0gMC4xKSArCiAgZ2d0aXRsZSgiRGlhbW9uZHMgRGF0YSIpICsKICB4bGFiKCJDYXJhdCIpICsKICB5bGFiKCJQcmljZSIpICsKICBmYWNldF9ncmlkKCBjdXQgfiBjb2xvcikKCmBgYAoKCgoKCgojIyBTaWRlIE5vdGUgYWJvdXQgcGxhY2VtZW50IG9mIGBhZXNgCgpgYWVzYCBjYW4gZWl0aGVyIGdvIGluc2lkZSB0aGUgYGdncGxvdCgpYCBmdW5jdGlvbiwgb3IgaW5zaWRlIHRoZSBgZ2VvbV9bY2hhcnRdKClgIGZ1bmN0aW9uIGl0c2VsZiwgb3IgYm90aC4gVGhlIDMgZm9sbG93aW5nIG9wdGlvbnMgY3JlYXRlIHRoZSBzYW1lIHBsb3RzLCBidXQgdGhlIGNvZGUgaXMgc2xpZ2h0bHkgZGlmZmVyZW50LiAKCmBgYHtyLCB3YXJuaW5nPUYsIG1lc3NhZ2UgPSBGfQojb3B0aW9uIDEKZ2dwbG90KGRhdGEgPSBkaWFtb25kcywgKSArCiAgZ2VvbV9wb2ludChhZXMoeCA9IGNhcmF0LCB5ID0gcHJpY2UsIGNvbG9yID0gY2xhcml0eSksCiAgICAgICAgICAgICBhbHBoYSA9IDAuMiwgCiAgICAgICAgICAgICBzaXplID0gMSkgKwogIGdlb21fc21vb3RoKG1ldGhvZCA9ICJnbG0iICwgCiAgICAgICAgICAgICAgZm9ybXVsYSA9IHkgfiBwb2x5KHgsIDIpLCAgICAgICAgICAgICAgICAgICAgICMgeSA9IGJfMCArIGJfMSB4ICsgYl8yIHheMiArIGUKICAgICAgICAgICAgICBhZXMoeCA9IGNhcmF0LCB5ID0gcHJpY2UpLCAKICAgICAgICAgICAgICBjb2xvciA9ICJyZWQiKSArCiAgeWxpbShjKDAsIDIwMDAwKSkKICAKCgojb3B0aW9uIDIKZ2dwbG90KGRhdGEgPSBkaWFtb25kcywgYWVzKHggPSBjYXJhdCwgeSA9IHByaWNlLCBjb2xvciA9IGNsYXJpdHkpKSArCiAgZ2VvbV9wb2ludChhbHBoYSA9IDAuMiwgCiAgICAgICAgICAgICBzaXplID0gMSkgKwogIGdlb21fc21vb3RoKG1ldGhvZCA9ICJnbG0iICwgCiAgICAgICAgICAgICAgZm9ybXVsYSA9IHkgfiBwb2x5KHgsIDIpLCAgICAgICAgICAgICAgICAgICAgICMgeSA9IGJfMCArIGJfMSB4ICsgYl8yIHheMiArIGUKICAgICAgICAgICAgICBhZXMoeCA9IGNhcmF0LCB5ID0gcHJpY2UpLCAKICAgICAgICAgICAgICBjb2xvciA9ICJyZWQiKSArCiAgeWxpbShjKDAsIDIwMDAwKSkKICAKCiNPcHRpb24gMwpnZ3Bsb3QoZGF0YSA9IGRpYW1vbmRzLCAgYWVzKHggPSBjYXJhdCwgeSA9IHByaWNlKSApKyAKICBnZW9tX3BvaW50KCBhZXMoY29sb3IgPSBjbGFyaXR5KSwKICAgICAgICAgICAgICBhbHBoYSA9IDAuMiwgCiAgICAgICAgICAgICAgc2l6ZSA9IDEpICsKICBnZW9tX3Ntb290aChtZXRob2QgPSAiZ2xtIiAsIAogICAgICAgICAgICAgIGZvcm11bGEgPSB5IH4gcG9seSh4LCAyKSwgICAgICAgICAgICAgICAgICAgICAjIHkgPSBiXzAgKyBiXzEgeCArIGJfMiB4XjIgKyBlCiAgICAgICAgICAgICAgY29sb3IgPSAicmVkIikgKwogIHlsaW0oYygwLCAyMDAwMCkpCgpgYGAKCgotIEkgcGVyc29uYWxseSBwcmVmZXIgdG8gcHV0ICJnbG9iYWwiIGFlc3RoZXRpY3MgaW4gdGhlIGBnZ3Bsb3QoKWAgYW5kICJsb2NhbCIgYWVzdGhldGljcyBpbiB0aGUgYGdlb21gLiAKICAtIE9wdGlvbiAxIDogYWxsIGFlc3RoZXRpY3MgYXJlIGxvY2FsIHRvIHRoZSBnZW9tCiAgICAgLSBOb3RlIHdlIGhhdmUgdG8gcmVwZWF0IGB4YCBhbmQgYHlgCiAgLSBPcHRpb24gMiA6IGFsbCBhZXN0aGV0aWNzIGFyZSBnbG9iYWwgdG8gdGhlIGdncGxvdAogICAgLSBOb3RlIHRoYXQgYGNvbG9yID0gY2xhcml0eWAgaXMgbm90IG5lZWRlZCBmb3IgYGdlb21fc21vb3RoYAogIC0gT3B0aW9uIDMgOiBnbG9iYWwgYWVzdGhldGljcyBhcmUgaW4gdGhlIGdncGxvdCBhbmQgbG9jYWwgYWVzdGhldGljcyBhbmQgaW4gdGhlIGdlb20KICAgIC0gQm90aCBgZ2VvbV9wb2ludGAgYW5kIGBnZW9tX3Ntb290aGAgdXNlIGB4YCBhbmQgYHlgIHNvIEkgcHV0IHRoZW0gaW4gdGhlIGBnZ3Bsb3QoKWAKICAgIC0gT25seSBgZ2VvbV9wb2ludGAgdXNlcyBgY29sb3IgPSBjbGFyaXR5YCBzbyBJIHB1dCB0aGF0IE9OTFkgaW4gdGhlIGBnZW9tX3BvaW50YCBmdW5jdGlvbgogICAgCi0gSW4gbXkgb3BpbmlvbiwgT3B0aW9uIDMgaXMgdGhlICJjbGVhbmVzdCIgY29kZS4gVGhpcyBpcyBwYXJ0bHkgYmFzZWQgb24gc3R5bGlzdGljIHByZWZlcmVuY2UgYW5kIHBhcnRseSBiYXNlZCBvbiBzb21lIGludGVybmFsIG1lY2hhbmljIG9mIGdncGxvdCdzICh0aGF0IGlzIGJleW9uZCB0aGUgc2NvcGUgb2YgdGhpcyBjb3Vyc2UpLiBIb3cgeW91IHdyaXRlIHlvdXIgY29kZSBpcyB1cCB0byB5b3UuIEp1c3Qga2VlcCBpdCByZWFkYWJsZSEKCi0gQnV0IGFnYWluLCBhbGwgMyBjb2RlcyBnZW5lcmF0ZSB0aGUgdGhlIGV4YWN0IHNhbWUgcGxvdCAoc28gZG9lcyBpdCByZWFsbHkgbWF0dGVyIHRoYXQgbXVjaCB3aGljaCBvcHRpb24gd2UgdXNlPz8pCgo=